Evaluation Algorithms for Event Nugget Detection : A Pilot Study
نویسندگان
چکیده
Event Mention detection is the first step in textual event understanding. Proper evaluation is important for modern natural language processing tasks. In this paper, we present our evaluation algorithm and results during the Event Mention Evaluation pilot study. We analyze the problems of evaluating multiple event mention attributes and discontinuous event mention spans. In addition, we identify a few limitations in the evaluation algorithm used for the pilot task and propose some potential improvements.
منابع مشابه
Event Nugget and Event Coreference Annotation
In this paper, we describe the event nugget annotation created in support of the pilot Event Nugget Detection evaluation in 2014 and in support of the Event Nugget Detection and Coreference open evaluation in 2015, which was one of the Knowledge Base Population tracks within the NIST Text Analysis Conference. We present the data volume annotated for both training and evaluation data for the 201...
متن کاملNew York University 2016 System for KBP Event Nugget: A Deep Learning Approach
This is the first time New York University (NYU) participates in the event nugget (EN) evaluation of the Text Analysis Conference (TAC). We developed EN systems for both subtasks of event nugget, i.e, EN Task 1: Event Nugget Detection and EN Task 2: Event Nugget Detection and Coreference. The systems are mainly based on our recent research on deep learning for event detection (Nguyen and Grishm...
متن کاملEvent Argument Linking and Event Nugget Detection Task: IHMC DISCERN System Report
We present the design and implementation of three variants of an event-detection system called DISCERN, one that used manually created rules (DISCERN-R), one that used machine-learned rules (DISCERN-ML), and one that combined manual and machine-learned rules (DISCERN-C). All three used linguistic resources (VerbNet, CatVar, Semantic Role Labeling, NER, POS tagging, dependency parsing, and coref...
متن کاملEvent Nugget Detection Task : UMBC systems
This paper described our Event Nugget Detection system that we submitted to the TAC KBP 2016 Event Track. We sent out two runs; UMBC1 and UMBC2. UMBC1 is a sentencelevel classification system based on Convolution Neural Network and applied the probability to select a word as an event nugget. UMBC2 is the classification model trained from our features using Weka and filtered out low confidence p...
متن کاملHITS at TAC KBP 2015: Entity Discovery and Linking, and Event Nugget Detection
HITS participated in the English Entity Linking and Discovery (EDL) and Event Nugget Detection (EN task 1) tracks at TAC KBP 2015. Our EDL system introduces a novel, interleaved multitasking approach, which allows interaction between interdependent entity linking subtasks while avoiding the structural and algorithmic complexity of joint models. Out of the eight systems that participated in the ...
متن کامل